In [15]:
%matplotlib inline
fig = exp.as_pyplot_figure()
In this series of notebooks, we will see how the lime module can be used to implement the methodology described in Ribeiro et al (2016) "Why Should I Trust You?: Explaining the Perdictions of Any Classifier". We will use the implementation of LIME in python to explain individual predictions from several classifiers.
These walk-throughs are adapted from the tutorials created by the package developers, which are available on their github page.
In this first walk-through we will explore how LIME can be used to interpret a classifier for text documents when there are only two document classed. In the second walk-though we will extend this to consider multiple document classes. In the third and final walk-through, we will look at some more conventional, tabular data that have numeric values.
If you would like to explore these ideas on other types of data, such as images, tutorials for these explainers can be found on the github page linked above.
This notebook has several dependancies, which can be loaded as follows:
import lime
import sklearn
import numpy as np
import sklearn
import sklearn.ensemble
import sklearn.metrics
from __future__ import print_function
In this first walk-through we will be using the 20 newsgroups dataset. The 20 newsgroups dataset comprises around 18000 newsgroups posts on 20 topics split in two subsets: one for training (or development) of a model and the other one for testing (or for performance evaluation) of the model. The split between the train and test set is based upon whether a message is posted before or after a specific date.
The 20 newsgroups dataset is included in scikit-learn (sklearn) and, in this walk-through, we will focus on only two of the categories: alt.atheism and soc.religion.christian. We can use the following code to load and store both the training and test data relating to these categories. We also create some more readable names for the two classes of document.
from sklearn.datasets import fetch_20newsgroups
categories = ['alt.atheism', 'soc.religion.christian']
class_names = ['atheism', 'christian']
newsgroups_train = fetch_20newsgroups(subset='train', categories=categories)
newsgroups_test = fetch_20newsgroups(subset='test', categories=categories)
Print element 0 of the training data newsgroups_train.data. What form does this take?
Print the training labels newsgroup_train.target. What form do these take?
How many training examples are there on the topic of Atheism or Christianity?
print(newsgroups_train.data[0])
From: nigel.allen@canrem.com (Nigel Allen)
Subject: library of congress to host dead sea scroll symposium april 21-22
Lines: 96
Library of Congress to Host Dead Sea Scroll Symposium April 21-22
To: National and Assignment desks, Daybook Editor
Contact: John Sullivan, 202-707-9216, or Lucy Suddreth, 202-707-9191
both of the Library of Congress
WASHINGTON, April 19 -- A symposium on the Dead Sea
Scrolls will be held at the Library of Congress on Wednesday,
April 21, and Thursday, April 22. The two-day program, cosponsored
by the library and Baltimore Hebrew University, with additional
support from the Project Judaica Foundation, will be held in the
library's Mumford Room, sixth floor, Madison Building.
Seating is limited, and admission to any session of the symposium
must be requested in writing (see Note A).
The symposium will be held one week before the public opening of a
major exhibition, "Scrolls from the Dead Sea: The Ancient Library of
Qumran and Modern Scholarship," that opens at the Library of Congress
on April 29. On view will be fragmentary scrolls and archaeological
artifacts excavated at Qumran, on loan from the Israel Antiquities
Authority. Approximately 50 items from Library of Congress special
collections will augment these materials. The exhibition, on view in
the Madison Gallery, through Aug. 1, is made possible by a generous
gift from the Project Judaica Foundation of Washington, D.C.
The Dead Sea Scrolls have been the focus of public and scholarly
interest since 1947, when they were discovered in the desert 13 miles
east of Jerusalem. The symposium will explore the origin and meaning
of the scrolls and current scholarship. Scholars from diverse
academic backgrounds and religious affiliations, will offer their
disparate views, ensuring a lively discussion.
The symposium schedule includes opening remarks on April 21, at
2 p.m., by Librarian of Congress James H. Billington, and by
Dr. Norma Furst, president, Baltimore Hebrew University. Co-chairing
the symposium are Joseph Baumgarten, professor of Rabbinic Literature
and Institutions, Baltimore Hebrew University and Michael Grunberger,
head, Hebraic Section, Library of Congress.
Geza Vermes, professor emeritus of Jewish studies, Oxford
University, will give the keynote address on the current state of
scroll research, focusing on where we stand today. On the second
day, the closing address will be given by Shmaryahu Talmon, who will
propose a research agenda, picking up the theme of how the Qumran
studies might proceed.
On Wednesday, April 21, other speakers will include:
-- Eugene Ulrich, professor of Hebrew Scriptures, University of
Notre Dame and chief editor, Biblical Scrolls from Qumran, on "The
Bible at Qumran;"
-- Michael Stone, National Endowment for the Humanities
distinguished visiting professor of religious studies, University of
Richmond, on "The Dead Sea Scrolls and the Pseudepigrapha."
-- From 5 p.m. to 6:30 p.m. a special preview of the exhibition
will be given to symposium participants and guests.
On Thursday, April 22, beginning at 9 a.m., speakers will include:
-- Magen Broshi, curator, shrine of the Book, Israel Museum,
Jerusalem, on "Qumran: The Archaeological Evidence;"
-- P. Kyle McCarter, Albright professor of Biblical and ancient
near Eastern studies, The Johns Hopkins University, on "The Copper
Scroll;"
-- Lawrence H. Schiffman, professor of Hebrew and Judaic studies,
New York University, on "The Dead Sea Scrolls and the History of
Judaism;" and
-- James VanderKam, professor of theology, University of Notre
Dame, on "Messianism in the Scrolls and in Early Christianity."
The Thursday afternoon sessions, at 1:30 p.m., include:
-- Devorah Dimant, associate professor of Bible and Ancient Jewish
Thought, University of Haifa, on "Qumran Manuscripts: Library of a
Jewish Community;"
-- Norman Golb, Rosenberger professor of Jewish history and
civilization, Oriental Institute, University of Chicago, on "The
Current Status of the Jerusalem Origin of the Scrolls;"
-- Shmaryahu Talmon, J.L. Magnas professor emeritus of Biblical
studies, Hebrew University, Jerusalem, on "The Essential 'Commune of
the Renewed Covenant': How Should Qumran Studies Proceed?" will close
the symposium.
There will be ample time for question and answer periods at the
end of each session.
Also on Wednesday, April 21, at 11 a.m.:
The Library of Congress and The Israel Antiquities Authority
will hold a lecture by Esther Boyd-Alkalay, consulting conservator,
Israel Antiquities Authority, on "Preserving the Dead Sea Scrolls"
in the Mumford Room, LM-649, James Madison Memorial Building, The
Library of Congress, 101 Independence Ave., S.E., Washington, D.C.
------
NOTE A: For more information about admission to the symposium,
please contact, in writing, Dr. Michael Grunberger, head, Hebraic
Section, African and Middle Eastern Division, Library of Congress,
Washington, D.C. 20540.
-30-
--
Canada Remote Systems - Toronto, Ontario
416-629-7000/629-7044
print(newsgroups_train.target[0])
1
print(len(newsgroups_train.data))
# or
#print(len(newsgroups_train.target))
1079
The training data are text strings, contatining meta-information about each posting in a header and a footer, as well as the body of the message.
The target variable is a binary variable corresponding to which element of class_namesthat document belongs, so that 0 = atheist and 1 = christian.
The training data set contains 1079 documents that are on the topic of atheism or christianity.
To turn the text strings into something more ammenable to computation we will apply the tfidf vectoriser. This converts the string into a numeric vector.
Search online to find what the acronym tfidf stands for.
tfidf stands for "term frequency–inverse document frequency".
For a single word, the tfidf value is a statistic that aims to compare word importance in a particular document to its importance in a collection of documents. It does so by comparing how many times the word appears in that document relative to how many times it occurs in the corpus (collection of documents) that includes the document of interest. A tfidf vector produces such a summary statistic for each word in the corpus.
Find out how the tfidf vectoriser converts a string to a numeric vector.
There are several methods of calculating the tfidf vector, see this wikipedia article for more details. The vectoriser that we are using (see help pages here) defines, $n$ to be the number of documents in the corpus. Then for word $w$ and document $d$, the term frequency to be $\text{tf}(w,d)$, the number of times word $w$ appearsin document $d$. The document frequency $\text{df}(w)$ is the number of documents in the corpus containing $w$. The inverse document frequency is then given by:
$$ \text{idf}(w) = 1 + \log \left( \frac{1 + n}{1 _ \text{df}(w)} \right).$$From these quantities, the tf-idf can then calculated for each word-document pair as:
$$\text{tf-idf}(w,d) = \text{tf}(w,d) \times \text{idf}(w).$$To apply this transformation to our training and test data, we first have to create a vectorizer object and then use this to create our training and test vectors. _Note that we use vectorizer.fit_transform for the training data and vectorizer.transform for the test data._
vectorizer = sklearn.feature_extraction.text.TfidfVectorizer(lowercase=False)
train_vectors = vectorizer.fit_transform(newsgroups_train.data)
test_vectors = vectorizer.transform(newsgroups_test.data)
Now that we have our training and test data in a suitable form, we can fit a random forest to the training data and use it to predict the classes of the test data.
You may or may not already know what a random forest is, either way is perfectly fine. You will learn more about such classification methods later this year during the supervised learning course. If you already know about random forests, then you will be aware that it can be very difficult to understand how a fitted random forest model obtains its predictions. This makes predictions based on this type of model difficult to explain to a client, a stakeholder or a (dis-)satisfied customer. On the other hand, if you do not yet know about random forests then you are in the same position as that client, stakeholder or customer. You are in the perfect place to see how LIME can provide an intuition for the a prediction made by a random forest.
Fitting the random forest is simple using the functionality built into sklearn.
rf = sklearn.ensemble.RandomForestClassifier(n_estimators=500)
rf.fit(train_vectors, newsgroups_train.target)
RandomForestClassifier(n_estimators=500)
We can now use the trained random forest classifier to predict the class of each document in the test set; namely whether it relates to Christian or Atheist content. We can evaluate the quality of these predictions using the F-score, which takes values between 0 and 1.
pred = rf.predict(test_vectors)
sklearn.metrics.f1_score(newsgroups_test.target, pred, average ='binary')
0.9274004683840751
Calculate the proportion of the test documents that our random forest classifier predicts correctly.
accuracy_rf = sum(pred == newsgroups_test.target) / len(pred)
print(accuracy_rf)
0.9135285913528591
Why does this differ from the F score?
Our previous statistic cared only about the accuracy of the classifier (proportion of test documents correctly classified). The F statistic also measures the recall of the classifier (proportion of test documents classified as Christian that are actually Christian). The F score combines these two properties through thier harmonic mean:
$$F = \frac{2}{\text{precision}^{-1} + \text{recall}^{-1}} = \frac{\#(\text{true Christian predictions})}{\#(\text{true Christian predictions}) + 0.5(\#(\text{false predictions}))}.$$The sklearn page on the 20 newsgroups data shows that another, easily explained, classifier overfits this dataset by basing its predictions on irrelevant parts of the text such as headers. We will use LIME to see if random forests do the same.
The lime module explainer functions for text data assumes that the classifier acts on raw text. However, we have used a random forest classifier from sklearn, which used a vectorised form of the text. To get around this issue, we will create a pipleline (a functionality from sklearn to combine multiple operations) to implement predict_proba on raw_text lists.
We can then use this pipeline to find the probability that each test document belongs to the class atheism or christian.
from sklearn.pipeline import make_pipeline
c = make_pipeline(vectorizer, rf)
print(c.predict_proba([newsgroups_test.data[0]]))
[[0.274 0.726]]
What is the estimated probability that the final test document is christian? What is the estimated probability that the penultimate test document is about atheism?
print(c.predict_proba([newsgroups_test.data[-1]]))
print(c.predict_proba([newsgroups_test.data[-2]]))
[[0.212 0.788]] [[0.096 0.904]]
The random forest classifier predicts that the final document belongs to the christian class with probability 0.802. Similarly, the penultimate document is predicted to belong to the atheism class with probability 0.094.
We now use LIME to explain these predictions. To do this we create an explainer object from the lime module. To make the resulting output prettier, we pass this object the vector of class_names that we created at the start of this walk-through.
from lime import lime_text
from lime.lime_text import LimeTextExplainer
explainer = LimeTextExplainer(class_names=class_names)
We can then use this explainer object to generate an explanation for any document in the test set. In doing this, we will limit ourselves to having at most 6 explaining features in any explanation.
index = 1
exp = explainer.explain_instance(newsgroups_test.data[index], c.predict_proba, num_features=6)
print('Document id: %d' % index)
print('Predicted class:', class_names[c.predict([newsgroups_test.data[index]])[0]])
print('Probability(Christian) =', c.predict_proba([newsgroups_test.data[index]])[0,1])
print('True class: %s' % class_names[newsgroups_test.target[index]])
exp.as_list()
Document id: 1 Predicted class: christian Probability(Christian) = 0.71 True class: christian
[('morality', -0.04208051578624302),
('clh', 0.02117787802909364),
('alt', -0.019176994653475214),
('to', 0.01631619368118953),
('try', -0.016156940044239135),
('religious', -0.015237855182125844)]
The random forest correctly identified the first test document as being christian, and did so with fairly high probability.
The LIME explanation of this prediction is presented as a list of weighted features. These weighted features represents a linear model that approximates the local behaviour of the random forest classifier in the vicinity of the test document.
Roughly speaking, if we remove the words morality and alt from the document, then our predicted probability that this is a christian document should increase by between 5 and 6 percent (or more precisely by 0.05813). Note that this change is the sum of the explanation weights for those words, multiplied by -1 since those words are being removed from the text.
Recall that this local linear model is only an approximation. Let's check and see how much the probability of Christianity really changes when these words are removed.
print('Original prediction:', rf.predict_proba(test_vectors[index])[0,1])
tmp = test_vectors[index].copy()
tmp[0,vectorizer.vocabulary_['morality']] = 0 # set word count for 'morality' to 0
tmp[0,vectorizer.vocabulary_['alt']] = 0 # set word count for 'alt' to 0
print('Prediction removing "morality" and "alt":', rf.predict_proba(tmp)[0,1])
print('Difference in probability:', rf.predict_proba(tmp)[0,1] - rf.predict_proba(test_vectors[index])[0,1])
Original prediction: 0.71 Prediction removing "morality" and "alt": 0.772 Difference in probability: 0.062000000000000055
The predicted change from our LIME explainer is pretty close to the true change!
While some of the words that are inclided in the explainer seem topical, such as morality and atheist, others seem very arbitrary - the words clh,alt and and don't seem to have much to do with either Christianity or Atheism. We will investigate this further by visualising our explanations.
There are two main ways to visualise our explanations. The first is to return the explainer as a matplotlib barplot. This makes our list object much nicer to interpret.
%matplotlib inline
fig = exp.as_pyplot_figure()
Alternatively, we can export the visualisation as an hmtl page using D3.js to render the graphs. This html page can be rendered nicely in a notebook environment as follows.
exp.show_in_notebook(text=False)
Alternatively, it can be saved as a stand-alone html file.
exp.save_to_file('explainer_1.html')
Finally, we can also include a visualisation of the originial document, with the words in the explanation highlighted according to which class they suggest. Notice that not all of these words are particularly sensible but that alt, which seemed non-sensical when taken out of context, is actually a posting tag that may well generalise to classification of other posts.
exp.show_in_notebook(text=True)
Create and interpret a visualisation of the top 5 words explaining the final post in the test set.
task_index = len(newsgroups_test.data) - 1
task_exp = explainer.explain_instance(newsgroups_test.data[task_index], c.predict_proba, num_features=5)
print('Document id: %d' % task_index)
print('Predicted class:', class_names[c.predict([newsgroups_test.data[task_index]])[0]])
print('Probability(Christian) =', c.predict_proba([newsgroups_test.data[task_index]])[0,1])
print('True class: %s' % class_names[newsgroups_test.target[task_index]])
task_exp.as_list()
Document id: 716 Predicted class: christian Probability(Christian) = 0.788 True class: christian
[('morality', -0.035546100816578406),
('In', -0.03452229819320968),
('Christ', 0.02095946634785964),
('Christians', 0.019926299890703266),
('Jesus', 0.01902546362342829)]
task_exp.show_in_notebook(text=True)
The final post in the test dataset is predicted to be christian with probability 0.81. In the visualisation the LIME explanation we can see that the key words contributing to this classification are Christ, Christians and God, which all seem to be sensible indicators that this is a christian document. On the other hand, the words that suggest that this is a document about atheism include the very generic word In. This word has high influence suggesting that the random forest might be overfitting to the training data and classifying based on relationships that would not extend beyond this dataset.
Create and interpret a visualisation of the top 7 words explaining the post at index 300 in the test set.
task_index = 300
task_exp = explainer.explain_instance(newsgroups_test.data[task_index], c.predict_proba, num_features=7)
print('Document id: %d' % task_index)
print('Predicted class:', class_names[c.predict([newsgroups_test.data[task_index]])[0]])
print('Probability(Christian) =', c.predict_proba([newsgroups_test.data[task_index]])[0,1])
print('True class: %s' % class_names[newsgroups_test.target[task_index]])
task_exp.show_in_notebook(text=True)
Document id: 300 Predicted class: atheism Probability(Christian) = 0.404 True class: atheism
This example is classified as a document on atheism, but the prediction is made with low confidence. This example shows two postential issues with the our classifier and explainer.
Firstly, the top three words suggesting that this post is about atheism are in the header of the email and apparently have nothing to do with the email topic. Similarly, the word most strongly suggesting that this is a Christian document is the, which does not seem overly informative about Christian content. In this way, LIME has allowed us to identify that our random forest is using parts of the text that would likely not generalise beyond this data set when making its predictions.
Secondly, the remaining words that explain the athisim classification are both to do with Islam. While that may seem strange, it can be understood if we interpret these words as indicating that the document is very likely not to do with christianity, rather than actually being about atheism. Certainly, this classification leaves a lot to be desired.
Note that while we have used a random forest classifier in this example, the lime module will work with any classifier that has a predict_proba method. In the next walkthrough we will consider using the LIME explainer with a different classifier and a multi-class problem.
End of File.